Skip to main content

Reply to this post | See parent post | Go Back
View Post [edit]

Poster: Brad Leblanc Date: Oct 5, 2004 5:53am
Forum: etree Subject: Re: mkw errors - FLAC is Better

Sorry to nitpick as well, but SHN sometimes will give Decompression failure errors that prevent downloaders from decoding the files. If you could only see the list of error reports we have for SHN shows not decompressing... There's gotta be at least 15 or 20 of them.

Here's some examples (just click "See Reported Errors"):
http://www.archive.org/audio/etree-details-db.php?id=6284
http://www.archive.org/audio/audio-details-db.php?id=2729

The only fix is to track down the original seeder (hope they still have the original files) and convince them to re-encode the show to FLAC. otherwise the show cannot be decompressed by anyone.

Use FLAC folks. Despite being slower, it's much less buggy, has more features, and is not proprietary.

-Brad

This post was modified by Brad Leblanc on 2004-10-05 12:53:38

Reply to this post
Reply [edit]

Poster: ssamadhi97 Date: Oct 5, 2004 10:45am
Forum: etree Subject: Re: mkw errors - FLAC is Better

Brad, I downloaded several of the files that were claimed to be corrupt in the two filesets you linked to and found them to be "fine", like in "decodable; resulting wav file is compliant with RIFF wav specs; no glitches upon listening". I used two entirely different decoders so I can claim with high confidence that the files are perfectly fine - decoders used were Shorten.exe 3.6.0 and foo_shn 0.3.14 for foobar2000 (because it has its own decoder backend implementation).

Here's what I've found out:

The wavs in the KDTU show have an additional unnecessary "PAD " block between the wav "fmt " header chunk and the actual sample "data" block, valid, but apparently enough to throw off software that is supposed to convert wav to aiff but isn't "clever" enough to properly handle such cases (by discarding the unnecessary block)

FLAC is not a wav compressor, but an audio compressor - as such it will discard this "PAD " block and any other unnecessary non-audio blocks before encoding - hence writing a proper wav/aiff audio file is trivial after FLAC decompression. And hence people don't have problems with the file reencoded to FLAC.

Shorten on the other hand strives to conserve everything contained in the wav/aiff file that's being compressed - including auxiliary non-audio (meta)data. One side effect of this is described above, another one is that the Shorten plugin for WinAmp 2.0 ("ShnAmp") plays such files back with short periods of silence or clicks or other glitches between tracks. Which makes me wonder why the WinAmp/ShnAmp combo is recommended by archive.org for Shorten playback by the way.. but I digress.

One possible fix for shows that are marked as erroneous because of non-standard RIFF chunks like "PAD ", "cue ", etc would be re-encoding them to FLAC archive.org-side - of course after making sure that the shn files match the md5 hash and decode to wav without any errors.


As for the "premature eof" problems, I for one would blame this on failed download attempts. And yes, you'll get the same kind of error if you download only half of a FLAC file. ;)

This post was modified by ssamadhi97 on 2004-10-05 17:45:45

Reply to this post
Reply [edit]

Poster: Brad Leblanc Date: Oct 5, 2004 12:06pm
Forum: etree Subject: Re: mkw errors - FLAC is Better

As for the "premature eof" problems, I for one would blame this on failed download attempts. And yes, you'll get the same kind of error if you download only half of a FLAC file.

This is what I thought too, but many of the reports say stuff like "MD5 passes, I tried downloading it 3 times and the same error happens every time." Really? WTF? In addition, some have comments from other curators that they can confirm the error message - maybe the error can be avoided by ditching mkwACT...?

Anyway, the rest of your post was very informative, thanks for the info.

-Brad

This post was modified by Brad Leblanc on 2004-10-05 19:06:23

Reply to this post
Reply [edit]

Poster: ssamadhi97 Date: Oct 5, 2004 8:19pm
Forum: etree Subject: Re: mkw errors - FLAC is Better

No idea whether mkwACT is really to blame here.. I don't like that piece of software myself either, but based on what I've seen so far I won't judge it yet.

As for "md5 verification is successful, decompression fails", none of the sample tracks I grabbed yesterday exhibited this kind of behaviour.. guess I'll try downloading the entire DSO show you linked to and see what happens.

Reply to this post
Reply [edit]

Poster: ssamadhi97 Date: Oct 6, 2004 1:20am
Forum: etree Subject: Re: mkw errors - FLAC is Better

Well as far as the DSO show is concerned, "Several of the files check ok with their md5, but then don't expand correctly under shorten." is certainly a weird claim, which I wasn't able to verify.

I downloaded the entire show; verified the md5 hashes (with both fsum and md5summer) - all files matched the stored checksum; decoded the files (again with both Shorten.exe 3.6.0 and foobar2000/foo_shn 0.3.14) - NO errors whatsoever (like premature EOF) were reported.

Furthermore, I ran a bitwise comparison on the output of both decoders (using foo_bitcompare in foobar2000 to compare the wavs created with Shorten.exe with the foo_shn decoder output generated on the fly from the shn files), and found the resulting audio streams to be bitwise identical.

So it's pretty much 100% safe to say that these files are intact and any (non-broken) Shorten decoder should be able to decode them without problems.

Conclusion: "no problems here".. now what? Guess I'll go listen to my new show..

Reply to this post
Reply [edit]

Poster: Brad Leblanc Date: Oct 6, 2004 6:39am
Forum: etree Subject: Re: mkw errors - FLAC is Better

NO errors whatsoever (like premature EOF) were reported.

Good! Psyched we won't have to track down uploaders for some of these (hopefully won't have to track down any of them, but I need to test all of them).

...decoded the files (again with both Shorten.exe 3.6.0 and foobar2000/foo_shn 0.3.14)

When you say shorten.exe 3.6.0 I assume you mean that you just decoded the files via commandline using the latest version of shorten (3.6.0) - Right?

Foobar2000 isn't something I've played with yet, I'll give that a shot.

Maybe you're right that we should recommend something besides mkwACT - that piece of software was grandfathered in when the project started, and nobody has challenged it's status as "Best Tool for the Community" until now.

Again, thanks for your input here.

-Brad

This post was modified by Brad Leblanc on 2004-10-06 13:39:26

Reply to this post
Reply [edit]

Poster: ssamadhi97 Date: Oct 6, 2004 9:29am
Forum: etree Subject: Re: mkw errors - FLAC is Better

When you say shorten.exe 3.6.0 I assume you mean that you just decoded the files via commandline using the latest version of shorten (3.6.0) - Right?

Exactly.

Foobar2000 isn't something I've played with yet, I'll give that a shot.

Good, good :) A thought or two on this: it's perfect for Shorten playback; however (sort of like FLAC) it will discard / ignore all non-standard data chunks stored in the Shorten file when decoding.

What I'm getting at is that if you use the foobar diskwriter for decoding a Shorten file(set), you will get audio data that's bit-identical to the audio data of the original wavs (pre-Shorten-compression), but the auxiliary chunks will be missing. Pretty much the same deal as with going wav > FLAC > wav, with the sole difference being that this time around uninteresting data is discarded by the DEcoder (foo_shn), instead of the ENcoder (flac.exe).


As for those auxiliary chunks themselves, they are usually introduced by audio editing software like CoolEdit/Audition or SoundForge and may contain anything ranging from metadata information like artist, title, to cue sheets etc (though no software other than the software that wrote this data will make use of this, since there's no standardized way of storing it, not even a common nonstandard way like id3vX for mp3s) to useless things like a bunch of zero bytes.

Yes, you guessed it already, those chunks are by and large completely uninteresting and useless. ;)


Hope I didn't confuse you too much with this :)

This post was modified by ssamadhi97 on 2004-10-06 16:29:51

Reply to this post
Reply [edit]

Poster: ssamadhi97 Date: Oct 5, 2004 9:45am
Forum: etree Subject: Re: mkw errors - FLAC is Better

Well actually the Shorten file format is so simple in construction (dare I say braindead?) that I doubt that there's any chance for such an error to occur unless the file is corrupted (and bitstream corruption will break a FLAC file just as well)

Other than that, I won't argue.. FLAC is the better choice indeed.